A computing system, such as a general purpose computing platform, has various resources that are made available to software that executes on the computing system. Examples of the computing system resources include processor time, I/O bandwidth, memory space, and disk space. As used herein, the term “host system” refers to a computing system including various resources that may be shared by software that executes on the host system.
One type of software that may execute on a host system is virtualization software. A host system running virtualization software may be referred to as a virtual computer system. As described in existing literature, including prior patents and patent applications from VMware, Inc., the assignee of this patent application, virtual computer systems come in many different varieties, including some that are “hosted” and others that are “unhosted.” In the case of a hosted virtual computer system, a distinct host operating system may provide certain functionality, such as for interfacing with input/output (I/O) devices, for use by the core virtualization software and/or by other software executing in the computer system. In an unhosted virtual computer system, the core virtualization software itself may include some or all of the functionality provided by the host operating system in a hosted system. Virtual computer systems may involve full virtualization of a hardware architecture, or they may involve only partial virtualization, including so-called paravirtualized systems. Virtual computer systems may also support a virtual machine that has the same general hardware architecture as the underlying physical system, or they may support virtual machines that have a virtual hardware architecture that is substantially different from the underlying physical hardware architecture. Many other possibilities also exist or may be subsequently developed. This invention may be implemented in a wide variety of such virtual computer systems.
Virtualization software virtualizes the underlying resources of the host system such that an operating system and applications that run in a virtual machine only see virtual system resources that the virtualization software makes available in the virtual machine. The operating system running in a virtual machine is referred to herein as a “guest operating system,” while applications running in a virtual machine are generally referred to as “guest applications.” More generally, software running in a virtual machine, not including software that supports, or forms part of, the virtualization software, is generally referred to as “guest software.” The virtual system resources made available to the guest software may be a portion of the host system resources that are allocated to the virtual machine, such as a portion of the host memory, for example; or they may be emulated virtual system resources that do not directly correspond with host system resources, such as a physical USB (Universal Serial Bus) port, with a connected USB flash memory device, being emulated as an attached SCSI (Small Computer System Interface) hard drive, for example. The virtual machine is an abstraction supported by the virtualization software. The virtual machine appears to the guest operating system and guest applications as a physical host system.
Multiple virtual machines may execute on the same host system. One or more thin layers of software may operate between each virtual machine and the underlying host system hardware. In some implementations, this software is referred to as a virtual machine monitor (VMM) or a hypervisor. In this patent application, the software that supports such virtual machines will be referred to as “virtualization software.” The term “virtualization software” is used in a broad sense herein to generally refer to any and all software that supports the operation of one or more virtual machines, including, for example, a host operating system in a hosted virtual computer system, and also including software that runs within the virtual machine, which nonetheless supports the operation of the virtual machine. Virtualization software does not, however, include ordinary software that is commonly executed in physical computer systems that do not involve the use of virtual machines, such as ordinary operating systems, ordinary applications, ordinary drivers, etc., that can execute in ordinary, non-virtualized computer systems.
As will be described in greater detail below, one function of the virtualization software is to control host system resource allocation among virtual machines. As mentioned above, one host system resource that may be allocated is persistent storage, such as hard disk storage. A host system may have access to one or more physical disk drives or other persistent storage devices, locally attached to the host system or accessible to the host system via a network or some other interface. The description below focuses on physical disk storage, although it also applies to other types of persistent storage. On a host system with no virtualization, a host operating system can be used to divide a disk into logical portions referred to as partitions or volumes, depending on the operating system being used. Once the physical disk is partitioned into logical partitions or volumes, a file system may be created on one or more of the partitions. The file system typically encompasses a portion of physical disk space that is fixed at boot-up time. The size of the fixed portion is typically selected to accommodate the operating system and initial applications, and to allow for future growth. The file system includes data structures that define file formats and organizational structure.
In a virtual computer system supporting one or more virtual machines, each virtual machine may be given a fixed amount of the physical disk space of the host system in the form of a virtual disk or other virtual storage. One method for virtualizing disk space involves the use of one or more files within the file system of the host system, where the one or more files in the host system may be used to store the contents of a virtual disk in the virtual machine. The virtualization software emulates a virtual disk for use by guest software, but the data that is written to the virtual disk is stored in one or more files in the host file system. For example, a 10 gigabyte file may be created on the host system to support the emulation of a 10 gigabyte virtual disk in the virtual machine. The physical disk space used to store the data written to a virtual disk need not be contiguous. Instead, the virtualization software provides a mapping functionality between locations on the virtual disk, as they appear to the guest software, and corresponding locations in the physical disk space.
Each virtual machine in a virtual computer system may have its own guest operating system and its own file system(s). Like the case where no virtualization is present, the amount of disk space provided to each virtual machine at boot-up time is typically enough space to store the guest operating system and guest applications, plus some additional space to allow for future growth.
When a fixed amount of disk space is provided to a virtual machine at boot-up time and a file system is implemented within the disk space, it is difficult to later change the amount of disk space encompassed by the file system. For example, changing the amount of disk space would typically require modification of a file system's internal data structures. Since some file systems are proprietary, such modification might require reverse engineering. In addition, different operating systems and file systems may be loaded on different virtual machines, each requiring custom programming to modify the amount of disk space allocated to the respective file system.
In order to avoid having to change the disk space provided to virtual machines, sufficient disk space may be provided to each virtual machine at boot-up time to account for all anticipated future growth. For example, if a virtual machine requires 5 gigabytes of storage to accommodate its initial footprint, it may be desirable to allocate 10 gigabytes of storage to the virtual machine as an initial allocation to allow for 5 gigabytes of future growth. However, it is often difficult to predict the growth needs of an individual virtual machine. If the initial allocation is more than will be needed, the number of virtual machines that execute on the host computer system may be unnecessarily limited, and/or the amount of physical disk space required for the host computer system may be unnecessarily large. If the initial allocation is too small, the virtual machine may not be able to support a sufficient number of guest applications or other software.
Another option is to initially refrain from allocating all of the physical disk space required to store the contents of an entire virtual disk. In the example above, instead of creating a 10 gigabyte file in the host system for the emulation of a 10 gigabyte virtual disk, the file in the host system may be smaller than 10 gigabytes. For portions (or blocks) of a virtual disk that have not yet been written to, the virtualization software may defer allocating corresponding physical disk space until those portions are written to by guest software. As more of the virtual disk is written to, the size of the file increases to contain the increased data content of the virtual disk. Virtual storage for which all of the corresponding physical storage space is not initially allocated, so that there is no corresponding physical storage reserved for virtual storage that has not yet been written to, is referred to herein as “sparse” virtual storage.
With the use of sparse virtual storage, the storage resources of a host computer system may be overcommitted. In other words, the sum of all of the virtual storage space that is provided to virtual machines in a virtual computer system may exceed the physical storage space of the host computer system. For example, suppose that a virtual machine is presented with a 10 gigabyte virtual disk. If the guest software in the virtual machine is only presently using 2 gigabytes of this virtual disk, then the contents of the virtual disk may be contained in a file that only occupies about 2 gigabytes of disk space. The remaining 8 gigabytes of physical storage that has not yet been allocated to the virtual machine may be used for other purposes instead, such as to store the data of other virtual disks used by other virtual machines, including other sparse virtual disks.
Suppose now that each of 10 virtual machines is presented with its own 10-gigabyte virtual disk. Suppose further that the guest software in each virtual machine initially uses only 2 gigabytes of its respective virtual disk. Suppose further that the host system has only a 50-gigabyte disk drive. In this case, the host disk is overcommitted by 50 gigabytes. As the disk utilization of the virtual machines increases, the available physical storage space will gradually be utilized. At some point, the guest software in one of the virtual machines may issue a write operation that causes the physical disk capacity of the host system to be exceeded. When this occurs, the disk hardware reports a write error. If this write error propagates to the guest operating system, the guest operating system may interpret it as a write error due to a disk failure, because the guest operating system will still see available disk space on its own virtual disk. The guest operating system may reattempt the write operation, but the reattempt will also fail because the host disk is full. In this situation, the virtual machine may shut down or fail.
The various embodiments described herein may employ various computer-implemented operations involving data stored in computer systems. For example, these operations may require physical manipulation of physical quantities usually, though not necessarily, these quantities may take the form of electrical or magnetic signals or states where they, or representations of them, are capable of being stored, transferred, combined, compared, or otherwise manipulated. Further, such manipulations are often referred to in terms, such as producing, identifying, determining, or comparing. Any operations described herein that form part of one or more embodiments of the invention may be useful machine operations. In addition, one or more embodiments of the invention also relate to a device or an apparatus for performing these operations. The apparatus may be specially constructed for specific purposes, or it may be a general purpose computer selectively activated or configured by a computer program stored in the computer. In particular, various general purpose machines may be used with computer programs written in accordance with the teachings herein, or it may be more convenient to construct a more specialized apparatus to perform the operations.
The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.
One or more embodiments of the present invention may be implemented as one or more computer programs or as one or more computer program modules embodied in one or more computer readable media. The term computer readable medium refers to any data storage device that can store data which can thereafter be input to a computer system computer readable media may be based on any existing or subsequently developed technology for embodying computer programs in a manner that enables them to be read by a computer. Examples of a computer readable medium include a hard drive, network attached storage (NAS), read-only memory, random-access memory (e.g., a flash memory device), a CD (Compact Discs) CD-ROM, a CD-R, or a CD-RW, a DVD (Digital Versatile Disc), a magnetic tape, and other optical and non-optical data storage devices. The computer readable code can also be distributed over a network coupled computer system so that the computer readable code is stored and executed in a distributed fashion.
Although one or more embodiments of the present invention have been described in some detail for clarity of understanding, it will be apparent that certain changes and modifications may be made within the scope of the claims. Accordingly, the described embodiments are to be considered as illustrative and not restrictive, and the scope of the claims is not to be limited to details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and/or steps do not imply any particular order of operation, unless explicitly stated in the claims.
In addition, while described virtualization methods have generally assumed that virtual machines present interfaces consistent with a particular hardware system, persons of ordinary skill in the art will recognize that the methods described may be used in conjunction with virtualizations that do not correspond directly to any particular hardware system. Virtualization systems in accordance with the various embodiments, implemented as hosted embodiments, non-hosted embodiments, or as embodiments that tend to blur distinctions between the two, are all envisioned. Furthermore, various virtualization operations may be wholly or partially implemented in hardware. For example, a hardware implementation may employ a look-up table for modification of storage access requests to secure non-disk data.
Many variations, modifications, additions, and improvements are possible, regarding the degree of virtualization. The virtualization software can therefore include components of a host, console, or guest operating system that performs virtualization functions. Plural instances may be provided for components, operations or structures described herein as a single instance. Finally, boundaries between various components, operations and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within the scope of the invention(s). In general, structures and functionality presented as separate components in exemplary configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements may fall within the scope of the appended claims(s).